Area-efficient Neuromorphic Silicon Circuits and Architectures using Spatial and Spatio-Temporal Approaches

In the field of neuromorphic VLSI connectivity is a huge bottleneck in implementing brain-inspired circuits due to the large number of synapses needed for performing brain-like functions. (E.g. pattern recognition, classification, etc.). In this thesis I have addressed this problem using a two pronged approach namely spatial and temporal.Spatial: The real-estate occupied by silicon synapses have been an impediment to implementing neuromorphic circuits. In recent years, memristors have emerged as a nano-scale analog synapse. Furthermore, these nano-devices can be integrated on top of CMOS chips enabling the realization of dense neural networks. As a first step in realizing this vision, a programmable CMOS chip enabling direct integration of memristors was realized. In a collaborative MURI project, a CMOS memory platform was designed for the memristive memory array in a hybrid/3D architecture (CMOL architecture) and memristors were successfully integrated on top of it. After demonstrating feasibility of post-CMOS integration of memristors, a second design containing an array of spiking CMOS neurons was designed in a 5mm x 5mm chip in a 180nm CMOS process to explore the role of memristors as synapses in neuromorphic chips.8Temporal: While physical miniaturization by integrating memristors is one facet of realizing area-efficient neural networks, on-chip routing between silicon neurons prevents the complete realization of complex networks containing large number of neurons. A promising solution for the connectivity problem is to employ spatio-temporal coding to encode neuronal information in the time of arrival of the spikes. Temporal codes open up a whole new range of coding schemes which not only are energy efficient (computation with one spike) but also have much larger information capacity than their conventional counterparts. This can result in reducing the number of connections to do similar tasks with traditional rate-based methods.By choosing an efficient temporal coding scheme we developed a system architecture by which pattern classification can be done using a “Winners-share-all” instead of a “Winner-takes-all” mechanism. Winner-takes-all limits the code space to the number of output neurons, meaning n output neurons can only classify n pattern. In winners-share-all we exploit the code space provided by the temporal code by training different combination of k out of n neurons to fire together in response to different patterns. Optimal values of k in order to maximize information capacity using n output neurons were theoretically determined and utilized. An unsupervised network of 3 layers was trained to classify 14 patterns of 15 x 15 pixels while using only 6 output neurons to demonstrate the power of the technique. The reduction in the number of output neurons results in the reduction of number of training parameters and results in lower power, area and memory required for the same functionality

A neuromorphic systems approach to in-memory computing with non-ideal memristive devices: From mitigation to exploitation

Memristive devices represent a promising technology for building neuromorphic electronic systems. In addition to their compactness and non-volatility features, they are characterized by computationally relevant physical properties, such as state-dependence, non-linear conductance changes, and intrinsic variability in both their switching threshold and conductance values, that make them ideal devices for emulating the bio-physics of real synapses. In this paper we present a spiking neural network architecture that supports the use of memristive devices as synaptic elements, and propose mixed-signal analog-digital interfacing circuits which mitigate the effect of variability in their conductance values and exploit their variability in the switching threshold, for implementing stochastic learning. The effect of device variability is mitigated by using pairs of memristive devices configured in a complementary push-pull mechanism and interfaced to a current-mode normalizer circuit. The stochastic learning mechanism is obtained by mapping the desired change in synaptic weight into a corresponding switching probability that is derived from the intrinsic stochastic behavior of memristive devices. We demonstrate the features of the CMOS circuits and apply the architecture proposed to a standard neural network hand-written digit classification benchmark based on the MNIST data-set. We evaluate the performance of the approach proposed on this benchmark using behavioral-level spiking neural network simulation, showing both the effect of the reduction in conductance variability produced by the current-mode normalizer circuit, and the increase in performance as a function of the number of memristive devices used in each synapse.Comment: 13 pages, 12 figures, accepted for Faraday Discussion

NEUROTECH - A European Community of Experts on Neuromorphic Technologies

Neuromorphic Computing Technology (NCT) is becoming a reality in Europe thanks to a coordinated effort to unite the EU researchers and stakeholders interested in neuroscience, artificial intelligence, and nanoscale technologies

Error-triggered Three-Factor Learning Dynamics for Crossbar Arrays

Recent breakthroughs suggest that local, approximate gradient descent learning is compatible with Spiking Neural Networks (SNNs). Although SNNs can be scalably implemented using neuromorphic VLSI, an architecture that can learn in-situ as accurately as conventional processors is still missing. Here, we propose a subthreshold circuit architecture designed through insights obtained from machine learning and computational neuroscience that could achieve such accuracy. Using a surrogate gradient learning framework, we derive local, error-triggered learning dynamics compatible with crossbar arrays and the temporal dynamics of SNNs. The derivation reveals that circuits used for inference and training dynamics can be shared, which simplifies the circuit and suppresses the effects of fabrication mismatch. We present SPICE simulations on XFAB 180nm process, as well as large-scale simulations of the spiking neural networks on event-based benchmarks, including a gesture recognition task. Our results show that the number of updates can be reduced hundred-fold compared to the standard rule while achieving performances that are on par with the state-of-the-art

Online Training of Spiking Recurrent Neural Networks with Phase-Change Memory Synapses

Spiking recurrent neural networks (RNNs) are a promising tool for solving a wide variety of complex cognitive and motor tasks, due to their rich temporal dynamics and sparse processing. However training spiking RNNs on dedicated neuromorphic hardware is still an open challenge. This is due mainly to the lack of local, hardware-friendly learning mechanisms that can solve the temporal credit assignment problem and ensure stable network dynamics, even when the weight resolution is limited. These challenges are further accentuated, if one resorts to using memristive devices for in-memory computing to resolve the von-Neumann bottleneck problem, at the expense of a substantial increase in variability in both the computation and the working memory of the spiking RNNs. To address these challenges and enable online learning in memristive neuromorphic RNNs, we present a simulation framework of differential-architecture crossbar arrays based on an accurate and comprehensive Phase-Change Memory (PCM) device model. We train a spiking RNN whose weights are emulated in the presented simulation framework, using a recently proposed e-prop learning rule. Although e-prop locally approximates the ideal synaptic updates, it is difficult to implement the updates on the memristive substrate due to substantial PCM non-idealities. We compare several widely adapted weight update schemes that primarily aim to cope with these device non-idealities and demonstrate that accumulating gradients can enable online and efficient training of spiking RNN on memristive substrates

Reliability Analysis of Memristor Crossbar Routers: Collisions and On/off Ratio Requirement

Memristors are commonly used in crossbar arrays as “in-memory computing” elements to solve the von-Neumann bottleneck problem. However, they can also be used as “in-memory routing” elements to configure on-chip interconnection schemes and route signals among computing elements in configurable multi-core neuromorphic processors. While there has been a significant focus on the use of memristive devices as in-memory computing elements, to date, studies on the fundamental reliability properties of memristors as routing elements are still missing. In this paper, we analyze the reliability issues of using these devices in routing crossbar arrays, caused by sharing routing resources (collisions), and undesired pulses due to the leakage paths (on/off ratio requirement). We show that there is a trade-off between routing collision probability and the degree of connectivity (i.e., fan-in) of the receivers sharing routing channels. We provide specifications and guidelines based on a theoretical analysis for engineering the properties of memristive devices, and for designing routing systems based on memristor crossbars

Women in circuits and systems (WiCAS) and young professionals (YP) at ICECS 2020.

A 'Women in Circuits and Systems' (WiCAS) event and a 'Young Professionals' (YP) event took place during the 27th Institute of Electrical and Electronics Engineers (IEEE) International conference on electronics circuits and systems (ICECS), the flagship conference for the Region 8 of the IEEE Circuits and Systems Society (CASS). The WiCAS events traditionally aim to inspire and motivate both students and young professionals in the domain of circuits and systems to have efficient roles in their professions, by meeting successful female engineers and professors, through interesting technical and professional talks in fields of interest of CASS. The YP events usually include start-ups presentation, poster and demo sessions, aiming to provide a thrilling environment for early career researchers to present their work. Joining this event, young professionals have the opportunity to learn about state of the art and most advanced activities in the area of circuits and systems, meet and interact with their peers, receive feedback from internationally well-known experts in the CASS domain from both academia and industry